Microscopy system, microscopy method, and a method of treating an aneurysm

ABSTRACT

A microscopy system and method allow observing a fluorescent substance accumulated in a tissue. The tissue can be observed at a same time both with visible light and with fluorescent light. It is possible to observe a series of previously recorded fluorescent light images in superposition with the visible light images. An end of the series of images may be automatically determined. A thermal protective filter may be inserted into a beam path of an illuminating system at such automatically determined end of the series. Further, the fluorescent light image may be analyzed for identifying a coherent fluorescent portion thereof. A representation of a periphery line of the coherent portion may be generated, and depths profile data may be obtained only from the coherent portion. An illuminating light beam for exciting the fluorescence may be modulated for improving a contrast of fluorescent images.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates to a microscopy system and a microscopy methodwhich may be in particular used for observing an emission of fluorescentlight at wavelengths of the near infrared and/or infrared. Further theinvention relates to a method of treating an aneurysm.

Fluorescent substances and fluorescent dyes showing a fluorescence atwavelengths in a region of the near infrared or infrared are used inmedical applications for various purposes such as visualizing particulartypes of tissue, tissue structures and tissue functions. Herein, afluorescent substance or dye or a precursor of such fluorescentsubstance or dye is applied to a patient under examination. The dyeaccumulates in particular types of tissue and tissue structures, and byobservation of the fluorescent light such types of tissue and tissuestructures, respectively, may be visualized and identified by anobserver. Optical tools are used to visualize the some times weakintensities of fluorescent light.

2. Brief Description of Related Art

One example of a suitable fluorescent substance is indocyanine green(ICG). From the article by T. Kuroiwa et al., “Development and ClinicalApplication of Near-Infrared Surgical Microscope Preliminary Report”,Minim Invas Neurosurg 2001; 44: 240-242, there is known a method andsystem for observing the fluorescence of this substance. An excitationwavelength of the fluorescence of the substance is about 780 nm, and afluorescent emission wavelength is about 835 nm. For an observation of atissue in which ICG has accumulated by a microscope, the tissue isilluminated with light of a main wavelength of 800 nm of a laser lightsource or of a halogen lamp. In a beam path of the illuminating lightthere is positioned a band-pass filter which allows only light havingwavelengths between 760 nm and 810 nm which is light for exciting thefluorescence to pass there-through. The tissue is imaged onto a cameraby a microscopy optics wherein a further band-pass filter allowing onlylight having wavelengths between 820 nm and 920 nm which is fluorescentemission light of ICG to pass there-through. An observation of imagesdetected by the camera allows to identify those regions of the tissue inwhich the fluorescent substance has accumulated. It is, however, notpossible to perceive surrounding regions of this tissue. Suchsurrounding regions would emit visible light under a suitableillumination. However, such illumination with visible light is notpossible since the laser light source does not provide such light, andthe band-pass filter in the beam path of the light source blocks suchvisible light from incidence on the tissue. A surgeon performing asurgical treatment of the tissue region has to illuminate the tissueregion with visible light for perceiving an optical image of the tissueregion with visible light in a first step, and he has to observe thefluorescent image thereafter for perceiving the fluorescent light in asubsequent second step. Further, an illuminating light beam of the laserlight source and an illuminating beam for illuminating the tissue regionwith visible light are incident on the surface region under differentangles such that both light beams generate different types of shadows onthe tissue region. Such different shadows render it more difficult tocorrectly associate regions of the fluorescent image with regions of thevisible image of the tissue region.

Such procedure is complicated and requires a high power of concentrationof the observer since he must remember the image previously perceived atthe respective different type of illumination.

The article of T. Kuroiwa et al., discloses an application of the ICGfluorescence for obtaining information about deep-seated tissues. Theexcitation and emission wavelengths of ICG lie within the “opticalwindow” of tissue, where absorption attributable to endogenouschromophoreous is low. Near infrared light can therefore penetratetissue to a depth of several millimeters to a few centimeters. Accordingto the article, a near infrared fluorescence of vessels on the brainsurface was observed through the dura mater. The article concludes thata stronger fluorescence emitted from the ICG will be necessary for theapplication to have practical use even for imaging venous vessels.

A cerebral or intra-cranial aneurysm is a dilatation of an artery in thebrain that results from a weakening of the inner muscular layer of ablood vessel wall. The vessel develops a “blister-like” dilation thatcan become thin and rupture without warning.

The vascular dilation itself is referred to as an aneurysm sac, and anentrance area that leads from the parent artery to the aneurysm sac isreferred to as aneurysm neck. According to a conventional surgicaltechnique of medical therapy of the aneurysm, a clip is used to closethe aneurysm sac. Thereafter, the surgeon has to verify that theaneurysm sac is completely closed and that a sufficient vascular flow isguaranteed in the parent artery and surrounding vessels.

According to a conventional technique the vascular flow is assessed byx-ray angiographie. For performing x-ray angiography, the surgicalmicroscope which must be used for performing the surgery and positioningthe clip has to be dismounted first, the x-ray apparatus has to bemounted thereafter, and the surgeon has to leave the room while thex-ray images are recorded. Development and interpretation of the x-rayimage will take some ten minutes. Intra-operative x-ray angiography isoften not used in practice for blood flow assessment due to the longduration until interpretation of the results. Further, an imagingresolution achievable with x-ray angiography is not sufficient forassessing blood flow in small vessels. Further, a complication rateassociated with x-ray angiography is considered to be relatively highdue to the necessity of applying a corresponding contrast agent via acatheter into an artery of the patient.

SUMMARY OF THE INVENTION

It is an object of the present invention to provide a method of surgicaltreatment of an aneurysm and allowing intra-operative observation anddocumentation of blood flow in small arterial vessels.

It is a further object of the present invention to provide a microscopysystem and a corresponding microscopy method which improve a performanceof microscopic imaging in combination with a fluorescence at infrared ornear infrared wavelengths.

According to a first aspect of the invention there is provided a methodof treating an aneurysm of a patient, the method comprising clipping theaneurysm, injecting indocyanine green into the patient generating atleast one fluorescence image of at least one artery adjacent to theclipped aneurysm, and assessing vascular blood flow of the at least oneartery based on the at least one fluorescence image.

The method using ICG as a fluorescent substance for assessing the bloodflow is simple and offers real time information on the patency ofarterial and venous vessels of all relevant diameters including smallarteries such as arteries of a diameter less than 0.5 mm, that arevisible to the surgeons eye. An interpretation of the recorded imagesmay be finished within one minute or less. Since ICG is not metabolizedin the body and is excreted by the liver with a plasma half time ofthree to four minutes, the method may be repeated after five to tenminutes.

A detailed illustration of the method is given in the article of A.Raabe et al., Neurosurgery, 132, volume 52, number 1, January 2003(www.neurosurgery-online.com). The full disclosure of this article isincorporated herein by reference.

According to a second aspect of the invention there is provided amicroscopy system and method having an improved performance inmicroscopic imaging fluorescent light images at infrared or nearinfrared wavelengths.

According to a further embodiment the invention provides a microscopysystem for visualizing a fluorescence of indocyanine green (ICG) whereinthe microscopy system comprises a microscopy optics, a display systemand an illumination system. The microscopy optics comprises a first beampath for imaging an object region onto a light detecting component ofthe first camera using light of a first range of wavelengths comprisingan emission wavelength or fluorescent wavelength of indocyanine green.Thus, the first beam path is provided for generating fluorescent lightimages of the object region. The microscopy optics further comprises asecond beam path for providing a magnified representation of the objectregion with light of wavelengths of a second range of wavelengthsincluding at least visible light. Thus, the second beam path is providedfor generating images of the object region in a visible region of thespectrum.

The display system displays the fluorescent light images superimposedwith the visible light representations such that a user may perceiveboth the fluorescent light images and the visible light representationsat a same time.

The illumination system provides at least one illuminating light beamdirected to the object region. The at least one illuminating light beamincludes light of the visible portion of the spectrum, in particularblue light and/or yellow light, and the illuminating light beam alsoincludes light with an excitation wavelength of indocyanine green.

The inventors recognized that it is possible to observe the objectregion with both visible light and fluorescent light at a same time whenusing the system illustrated above. It is not necessary to perform aconventional switching between observations under visible light andobservations under fluorescent light. Further, the provision of thevisible light and the excitation light in a common illuminating lightbeam prevents different shadows generated with excitation light andvisible light on a structured object, and substantially the sameillumination conditions are provided for both the excitation light andthe visible light.

According to a preferred embodiment the at least one illuminating lightbeam is generated by one single light source generating both the visiblelight and the light of an excitation wavelength of indocyanine green.The light source may comprise light sources such as a xenon lamp or ahalogen lamp.

Herein, the inventors recognized that suitable light sources which areconventionally used for generating illuminating light beams of visiblelight are able to generate excitation light of indocyanine green with asufficient intensity for recording fluorescent light images of theobject region with a suitable camera. Provision of a separate lightsource, such as a laser light source, for excitation of the fluorescenceof the indocyanine green, as it was used in the prior art, is no longernecessary.

According to a preferred embodiment, a filter is disposed in a beam pathof the illuminating system which filter substantially blocks light atemission wavelengths of the fluorescent emission wavelengths ofindocyanine green. It is thus possible to record fluorescent images of ahigh contrast.

According to a preferred embodiment the microscopy system is used in themethod of treating the aneurysm. According to a further embodiment, theillumination system comprises a thermal protective filter eliminatinglight from the illuminating light beam having wavelengths such that theyare not perceived by the human eye or that they are perceived with onlya low efficiency and which mainly contribute to heating of the objectunder examination. Thus, an unnecessary heating of the object iseffectively prevented. The thermal protective filter may be removed fromthe beam path of the illumination system when fluorescent light imagesof the object are to be recorded. The filters of the illumination systemmay be transmissive filters or reflective filters.

According to a preferred embodiment the microscopy system is used in themethod of treating the aneurysm.

According to a further embodiment the invention provides a microscopysystem for visualizing a fluorescence of a fluorescent substance whereinthe microscopy system comprises a microscopy optics, an image memory anda display system. The microscopy optics again comprises a first beampath for imaging a fluorescent light image of the object region onto acamera, and a second beam path for providing a representation of theobject region with light including at least visible light.

The image memory is configured for storing image data representingfluorescent light images of the object region recorded during a timeinterval.

The display system is configured to display the visible light image ofthe object region superimposed with a series of images generated fromthe previously recorded and stored image data.

It is thus possible to display rapidly changing configurations of theobject region which are visible in the fluorescent image merely during ashort time. Such rapidly changing configurations in the image aredisplayed in the overlaying relation with the visible light image of theobject region at a time after the rapid change in the configuration hasalready happened. The rapidly changing configurations may be repeatedlydisplayed or they may be displayed in slow motion.

According to a preferred embodiment the microscopy system is used in themethod of treating the aneurysm.

For this purpose indocyanine green is applied to the patient byintravenous injection, and the process of the fluorescent substanceentering the parent and surrounding vessels is observed with the camera.The process of entering the vessels may take 0.5 to 2 s. Herein it isimportant that the blood with the fluorescent substance enters therelevant vessels at a substantially same time. If the fluorescence willbecome visible in a particular vessel later than in surrounding vesselsthis will be indicative that the particular vessel is partially blocked.

The above illustrated assessment of vascular flow may be easilyperformed by repeatedly displaying the image data recorded during therespective time interval and/or by displaying the image data in slowmotion and in superposition with the visible light image of the objectregion.

If the fluorescent substance accumulates in the aneurysm sac this willbe indicative of a non-complete blocking of the aneurysm sac with theclip.

Preferably, the microscopy system comprises a controller for analyzingthe stored images with respect to a beginning process of the fluorescentsubstance entering the relevant vessels. Such start of the process maybe derived from a time at which intensities of the fluorescent lightexceed a threshold at some locations in the images.

The controller may also determine an end of such process by analyzingchanges in fluorescent light intensities from image to image, and if asaturation of increasing intensities is detected, an end of the processof fluorescent substances entering the relevant vessel will be decided.The controller may then supply only a subset of the stored image data tothe display system. The subset will include those data recorded afterthe start and/or before the end of the process of fluorescent substancesentering the relevant vessels.

According to a preferred embodiment the microscopy system is used in themethod of treating the aneurysm.

According to a further embodiment the invention provides a microscopysystem for visualizing a fluorescent substance wherein the microscopysystem comprises a microscopy optics, an illumination system and adisplay system. The microscopy optics again comprises a first beam pathfor imaging a fluorescent light image of the object region onto acamera, and a second beam path for a magnified imaging of the objectregion with visible light. The display system again displays the visiblelight image of the object region superimposed with the fluorescent lightimage of the object region generated from image data recorded with thecamera.

The illumination system provides an illuminating light beam directed tothe object region wherein the illuminating light beam includes light atan excitation wavelength of the fluorescent substance. The illuminatingsystem comprises a light modulator for modulating an intensity of thelight at the excitation wavelengths of the fluorescent substance. Thisresults in a modulation of the fluorescence of the substance in time,and the fluorescent images detected by the camera also vary in time.Such known variation in time of the intensities is taken into accountwhen generating the fluorescent light images with a high contrast. Forinstance, locations in the images detected by the camera may bedetermined which show varying intensities. Only those locations may bethen represented with a higher intensity in the generated fluorescenceimage.

According to a preferred embodiment the microscopy system is used in themethod of treating the aneurysm.

According to a further embodiment the invention provides a microscopysystem comprising a microscopy optics, an illumination system and acontroller. The microscopy optics again comprises a first and a secondbeam path which may traverse a common objective lens of the microscopyoptics. The first beam path images the object region onto a camera forgenerating image data representing the object region. The illuminationsystem provides an illuminating light beam directed to the objectregion. A filter is disposed in a first position within the beam path ofthe illuminating light beam. The illuminating system further comprisesan actuator for changing the position of the filter from a secondposition in which the filter is not disposed within the beam path to thefirst position.

The controller is configured to analyze the image data recorded with thecamera, and to control the actuator for displacing the filter from thefirst position to the second position based on such analysis. Theanalysis may comprise a determination of light intensities at particularportions of the images recorded with the camera.

Preferably the filter is a filter of such type that it eliminates lighthaving wavelengths greater than a predetermined wavelength from theilluminating light beam. The predetermined wavelength is preferablygreater than 690 nm. Further, the predetermined wavelength is preferablysmaller than 800 nm.

With such system it is possible to provide two different illuminationmodes wherein the system is automatically switched from one mode to theother mode based on an analysis of the images of the object regiondetected by the camera.

A preferred application of such microscopy system is in combination withthe above-illustrated process of a fluorescent substance entering someparticular vessels of interest. An end of such process is automaticallydetected. It is then possible to operate the system in a firstillumination mode during the recording of the fluorescent light imageswherein the illuminating light beam includes infrared light forexcitation of the fluorescence. When the end of the process is detectedthe thermal protective filter is automatically disposed in the beam pathof the illuminating system such that the system is operated in a secondillumination mode in which thermal radiation including infrared light isremoved from the illuminating light beam for preventing unnecessarythermal heating of the inspected object region.

According to a preferred embodiment the microscopy system is used in themethod of treating the aneurysm.

According to a further embodiment the invention provides a microscopysystem comprising a microscopy optics having a first beam path forgenerating fluorescent light images of the object region and a secondbeam path for providing a representation of the object region as seenwith visible light, and a display system for displaying thepresentations generated from the fluorescent images in superpositionwith visible light representations of the object regions. The microscopysystem further comprises a controller which is configured to analyze thefluorescent light images for detecting coherent portions of the image inwhich intensities of the fluorescent light exceed a threshold. Thecontroller then generates image data representing the fluorescent lightimage such that only peripheral lines of the coherent portions arevisible in the representation. The user may then identify those portionsof the object region in which the fluorescent intensity exceeds thethreshold by perceiving the peripheral line. At the same time it ispossible for the user to perceive an interior of the coherent portion asa representation of the visible light image.

According to a preferred embodiment the microscopy system is used in themethod of treating the aneurysm.

According to a further embodiment the invention provides a microscopysystem having an interferometer apparatus for generating depth profiledata representing an intensity of radiation of an analyzing light beamreceived back from an object. The analyzing light beam is directed tothe object, wherein a location to which the analyzing light beam isdirected on the object may be selected by control of a beam scanner.

Generation of such depth profile data is conventionally time consumingand, in particular, it is conventionally necessary to generate depthprofile data at substantially all scanning positions within an objectfield of the microscopy system.

According to the invention, a microscopy system having an interferometerapparatus is provided, wherein the microscopy system comprises a camerafor recording an image of the object with fluorescent light, and whereina controller is provided for analyzing the recorded image and fordetermining an analyzing portion within the image. The analyzing portioncomprises at least one coherent portion of the image recorded with thecamera in which an intensity in the image exceeds a threshold. Thecontroller may control the interferometer apparatus such that the depthprofile data are only generated at locations within the analyzingregion.

It is thus possible that the controller selects from the fluorescentlight image a tissue type of interest for the particular application,such as cancerous tissue, in dependence of the fluorescent image. Thoseportions in the image in which the tissue type of interest is presentare used as the analyzing region, and depth profile data are generatedonly for locations within the analyzing region. In portions outside ofthe analyzing region depth profile data are not generated.

It is thus not necessary that the user has to determine those portionsat which depth profile data should be generated, and it is possible toobtain the depth profile data for portions of interest in a relativelyshort time.

Preferably, the interferometer apparatus for generating the depthprofile data comprises an optical coherence tomography (OCT) apparatus.

BRIEF DESCRIPTION OF THE DRAWINGS

The forgoing as well as other advantageous features of the inventionwill be more apparent from the following detailed description ofpreferred embodiments of the invention with reference to theaccompanying drawings. Herein

FIG. 1 shows a schematic representation of beam paths of a microscopysystem according to an embodiment of the present invention;

FIG. 2 shows transmission characteristics of filters which may be usedin the microscopy system according to FIG. 1;

FIG. 3 is a flowchart for illustrating a microscopy method according toan embodiment of the invention;

FIG. 4 is a schematic representation of a further embodiment of themicroscopy system according to the invention;

FIG. 5 is a schematic representation of images generated with themicroscopy system of FIG. 4; and

FIG. 6 is a detailed partial illustration of an interferometer apparatusprovided in the microscopy system according to FIG. 4.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS

FIG. 1 schematically shows a microscopy system 1 comprising a microscopyoptics 3 including an objective lens 5 having an optical axis 7. Anobject 9 to be inspected is disposed in an object plane of objectivelens 5. Light emanating from the object 9 is transformed by theobjective lens 5 to form a parallel beam 11 in which two zoom systems12, 13 are disposed at a distance from the optical axis 7. The zoomsystems 12, 13 use partial beams 14 and 15 of the parallel beam 11 andsupply the partial beams 14, 15 to oculars 16 and 17 through deflectingprisms (not shown in FIG. 1) of a body of a tube of the microscopysystem 1. A user may perceive a magnified representation of the object 9as an image when looking into the oculars 16, 17 with his left eye 18and right eye 19, respectively. The image perceived with the left eye 18corresponds to an image when looking onto the object under an angle αwith respect to the optical axis, and the image perceived with the righteye 19 corresponds to an image when looking onto the object 9 under anangle −α with respect to the optical axis 7, such that the userperceives a stereoscopic image of the object 9 with his both eyes 18,19.

A partially transmissive mirror 21 is disposed in the partial beam 15for branching off a portion of the light thereof as a beam 23. Beam 23is split with a further beam splitter 25 to form beams 27 and 29. Beam27 is supplied to a light sensitive element of a camera 32 through acamera adapter optics 31 such that the camera 32 detects an image of theobject 9 under an observation angle −α with respect to optical axis 7.The images detected with camera 32 are transmitted as image data througha data line 33 to a controller 35.

A beam 39 is branched off from partial beam 14 by a partiallytransmissive mirror 37. Beam 39 is supplied to a light sensitive elementof a further camera 43 through a camera adapter optics 41 such thatcamera 43 detects images of the object 9 under an observation angle αwith respect to the optical axis 7. The images detected by camera 43 aresupplied as image data to controller 35 through a data line 45. Thecontroller transmits the images detected by cameras 32, 43 as image datathrough a line 47 to a head mounted display 49 which is carried by auser of the microscopy system 1 at his head such that integrateddisplays of the head mounted display 49 which are schematicallyindicated with reference numerals 51 and 52 in FIG. 1 provide respectiveimages which may be perceived by the user with his left and right eyes,respectively.

Thus, a user who is not in a position of directly looking into theoculars 16, 17 may also perceive a stereoscopic representation of theobject 9 by using the head mounted display 49 by observingrepresentations of visible light images of object 9.

Beam 29 is supplied to a light sensitive element of a camera 55 througha camera adapter optics 53 such that camera 55 may detect an infraredimage of the object. A filter 57 is disposed in beam 29. A transmissioncharacteristic of filter 57 is schematically illustrated as a line 58 inFIG. 2 a. A vertical line 59 in FIG. 2 a further represents a maximum ofan excitation spectrum of the fluorescent substance indocyanine green(ICG) at 780 nm. A vertical line 60 in FIG. 2 a represents acorresponding maximum of a fluorescent emission spectrum of ICG at 835nm. The transmission characteristic 58 of filter 57 shows a threshold 61at about 810 nm. The filter 57 is substantially non-transmissive atwavelengths below the threshold, and the filter 57 is substantiallytransparent at wavelengths above the threshold. The camera 55 thusdetects images of the object 9 which represent a distribution of thefluorescent substance within the object 9, provided that thefluorescence of the substance is excited with an illumination system 63of the microscopy system 1 as further illustrated below.

The images detected with camera 55 are transmitted through a data line65 to the controller 35. The controller 35 transmits the images detectedwith camera 55 as image data through a data line 67 to a LCD display 69.LCD display 69 represents the image data as an image which issuperimposed with partial beam 15 by a collimating optics 70 and apartially transmissive mirror 68. The image of the display 69 may beperceived by the eye 19 of the user in superposition with the directoptical image of the object 9. The LCD display 69 represents theinfrared light intensity distribution detected by camera 55 in a visiblecolor such as green color. Green color may be advantageously used forrepresenting the infrared image since human tissue forming the object 9usually comprises green color in a relatively low amount.

The controller performs a data processing of the image data transmittedto display 51 of head mounted display 49 such that the display 51displays the infrared light images detected by camera 55 superimposedwith the visible light images detected by camera 32. Thus, the userwearing the display 49 at his head may also perceive with his right eyea superimposed representation of visible light images and infrared lightimages of the object.

Even though it is not illustrated in FIG. 1 for ease of reference andclarity, a further beam may be branched off from partial beam 14supplied to the left eye 18. Such further beam may be supplied to afurther infrared camera for generating images which are superimposedwith the partial beam 14 of visible light as illustrated above forpartial beam 15, LCD display 69, collimating optics 70 and partiallytransmissive mirror 68. The user will then perceive a stereoscopicinfrared light image of the object 9. Image data generated by suchadditional camera may also be supplied to display 52 of head mounteddisplay 49 such that also head mounted display 49 provides astereoscopic representation of a distribution of the fluorescentsubstance within the object 9.

The illumination system 63 comprises a halogen lamp 71 as a lightsource, a reflector 72 and a collimator 73, for generating a collimatedlight beam 74 which is directed through one or plural lenses 75 onto anentrance end 76 of an optical fiber bundle 77 for coupling light emittedfrom light source 71 into the fiber bundle 77. The light is transportedby fiber bundle 77 to a position close to the objective lens 5 andemanates from an exit end 78 of the fiber bundle 77. A collimatingoptics 79 is provided for shaping the emanating light to form anilluminating light beam 81 directed to object 9. The embodiment is notlimited to using a halogen lamp as a light source. Also other lightsources, such as a xenon lamp, may be used.

In the presentation of FIG. 1 the collimating optics 79 is positionedcomparatively close to the objective lens 5. Nonetheless theilluminating light beam 81 is oriented with a relatively high angle withrespect to optical axis 7 of objective lens 5. Such high angle betweenthe directions of the illuminating light beam 81 and the main axis 7 ofthe objective lens may result in an insufficient illumination of theobject. In particular in a situation where a surgical method isperformed in a deep hole provided in the human body, such as in a humanbrain for treating an aneurysm, it may not be possible to perceive boththe visible light image and the fluorescent light image in superpositionwith each other. For such and other applications it is preferred to usea configuration of the illuminating system in which the illuminatinglight beam is oriented with a low angle to the optical axis 7. This maybe conventionally achieved by including the objective lens in the beampath of the illuminating light beam, i.e. the illuminating light beampasses through the objective lens, or the objective lens is providedwith an aperture penetrated by the illuminating light beam.

The illuminating system 63 further comprises a filter plate 83 havingtwo filters 84 and 85 disposed adjacent to each other. An actuator 87controlled by controller 35 is provided for displacing the filter plate83 in a direction indicated by double arrow 88 in FIG. 1 such thatfilter 84 is disposed in beam 74 in a first position of plate 83 andthat filter 85 is disposed in beam 74 in a second position of plate 83.

A transmission characteristics of filter 84 is shown in FIG. 2 b as aline 89 and represented in table 1 below:

TABLE 1 T [%] λ [nm] <05 300-385 =50 395-410 >85 420-660 >70 420-770 =70779 =0.1 801 <0.01  810-1200

It is evident that filter 84 transmits visible light and light withwavelengths up to a flank 90 of 800 nm substantially completely and thatwavelengths above flank are substantially not transmitted. Filter 84 ispositioned in beam 74 in applications for observing the fluorescence ofthe fluorescent substance in the inspected object region 9. Flank 90 at800 nm is at a wavelength higher than the excitation wavelength 59 ofICG such that light beam 81 both excites the fluorescence detected bycamera 55, and illuminates the object region with visible light suchthat the visible light image of the object may be perceived by lookinginto the oculars 16, 17 and by observing the images detected by cameras32, 43.

In an application where it is not necessary to observe the fluorescence,the controller 35 drives the actuator 87 to displace filter plate 83such that filter 85 is disposed in beam 74. A transmissioncharacteristics of filter 84 is schematically illustrated in FIG. 2 c asa line 91 and represented in table 2 below:

TABLE 2 T [%] λ [nm] <05 300-385 =50 395-410 >85 420-660 >70 420-660 =50680-710 <05  720-1180

It is evident that filter 85 substantially transmits visible light up toa flank 93 of the transmission characteristics 91, and that the filteris substantially non-transmissive for light at wavelengths above flank93. Flank 93 is at about 710 nm. The filter 85 is used as a thermalprotective filter for eliminating radiation from the illuminating lightbeam 81 which would otherwise result in an unnecessary heating of theobject 9. The flank 93 is well below the maximum excitation wavelength59 of ICG even though an excitation spectrum of ICG also extends towavelengths below 710 nm. However, a fluorescence of the substance issubstantially not excited when filter 85 is disposed in beam 74.

The controller further comprises an image memory 95 for storing a seriesof images or image data representing such images detected by camera 55.The controller 35 is further configured to successively supply the imagedata stored in memory 95 to display 69 such that display 69 displays theimages previously detected by camera 55 as a time series of images. Theseries of images may be also supplied to the head mounted display 49.

The display of the time series of the images may be understood as a filmrepresentation of the infrared light images. Such representation will bepreferably used in a situation where a process of the fluorescentsubstance entering a vessel system allows to assess and verifystructures or functions of the vessel system. At a time after thefluorescent substance has entered the vessel system the detected imagewill be substantially stationary in time, and substantially noadditional information may be gained from further observation of theinfrared light image. Since such process of entering the vessel systemis of a relatively short duration of one to five seconds, a surgeonwould have to watch the process with highest concentration and memorizethe time dependence of the process for each vessel of interest. Thepossibility of storing the images detected by camera 55 during theprocess and the possibility to repeatedly display the stored images as afilm will help the surgeon in gaining a complete impression of theobserved process.

An embodiment of a method of operating the microscopy system 1 will beillustrated below with reference to the flowchart of FIG. 3. At a startof an imaging procedure the thermal protective filter 85 is disposed inthe beam path of the illumination system 63, and the controller waits instep S1 for a start button of a switch 97 or some other input meansbeing operated by the surgeon or his assistant. Preferably, the startbutton 97 will be operated shortly before or after the injection of thefluorescent substance into the patient. In step S3 the actuator removesthe thermal protective filter 85 from the beam path and insertsfluorescent imaging filter 84 in the beam path, and in S5 a counter n isreset. Thereafter, an image B(0) detected by camera 55 is stored asimage data in memory 95 (S7). This image is also transferred bycontroller 35 to display 69. Display 69 displays the image such that thesurgeon may perceive the image in superposition with the visible lightimage of the object 9 when looking into the ocular (S9). Thereaftercounter n is incremented (S11), a next image B(n) is received fromcamera 55 and stored in memory 95 (S13), and this image B(n) isvisualized by display 69 or 51 in step S15.

Since the vessel system under observation does not contain fluorescentsubstance at the start of the procedure, the first detected images B(n)show substantially no intensities of infrared light. The fluorescentsubstance propagates through the body of the patient and finally entersthe tissue region 9 in the object field of the microscopy objective 3such that the images B(n) show successively increasing infraredintensities. The controller 35 analyses the intensities of the imagesB(n) and compares the intensities in step S17 with a first predeterminedthreshold. If the intensity of the latest detected image B(n) is lessthan the first threshold, processing is continued with step S11. If theintensity of image B(n) is higher than the first threshold this isindicative of a point in time used as a start of a series of detectedimages which series will be repeatedly displayed to the user later. Thecurrent value of counter n is assigned to a variable nstart in step S19.

Thereafter the counter is incremented (S20), the next image B(n) isobtained and stored (S21) and displayed (S23). The controller 35compares in step S25 the intensity of the last detected image B(n) withthe intensity of the second last image B(n−1) and continues processingat step S20, if the difference between both intensities is higher than apredetermined second threshold value. The second threshold value ischosen such that the condition of step S25 will not be fulfilled shortlyafter the start of the substance entering the vessel system since theintensities will continuously increase at that time. At some later timethe concentration of the fluorescent substance will come close to asaturation, and differences between intensities of subsequent imagesB(n) and B(n−1) will become smaller than the second threshold value.This is indicative of a point in time at which the series of detectedimages should be terminated. The current value of the counter n isassigned to a variable nend in step S27, and the fluorescent imagingfilter 83 is removed from the beam 74 and the thermal protective filter85 is inserted in beam 74 in step S29.

Thereafter the processing is continued by repeatedly displaying thedetected images with displays 69 or 51. For this purpose the counter nis set to the value nstart corresponding to the start of the series ofimages (S31), in which B(n) is displayed (S33), and the counter n isincremented (S35). If a present value of counter n is below the valuenend thereof at the end of the series of images in a step S37,processing is continued at step S33. Otherwise it is decided in step S39whether button 97 was again operated for indicating an end of theprocedure (S39). If the end of the procedure is not indicated,processing is continued at step S31 for displaying the series of imagesagain.

While the above illustrated embodiment uses one single light source forgenerating the light beam illuminating the object with visible light andfor exciting the fluorescence, a further embodiment may use differentlight sources for generating the visible light and the excitation light,respectively. The light source generating the excitation light may thenbe switched on and off according to the application.

A further embodiment may include two or plural light beams generated byone single light source or separate light sources wherein each beamincludes the visible light and the excitation light.

In the embodiment illustrated above with reference to FIGS. 1 to 3 themicroscopy system 1 uses filters 84 and 85 which are transmissivefilters. An alternative embodiment may use corresponding reflectivefilters which may be provided by a suitable coating of reflector 72 ofthe light source or which may be provided by separate reflectors withdesired filter characteristics which separated reflectors may beinserted in the beam path with an actuator under control of controller35.

In the embodiment illustrated above with reference to FIGS. 1 to 3 themicroscopy system 1 with its filters 57, 84 and 85 and filtercharacteristics shown in FIGS. 2 a, 2 b, 2 c, respectively, is optimizedfor observing the fluorescence of indocyanine green. In alternativeembodiments the above illustrated principles may be applied to anobservation of alternative fluorescent substances by adapting edges 90and 61 of characteristics 89 and 58 to the corresponding excitationwavelengths and fluorescent wavelengths of the alternative fluorescentsubstance.

FIG. 4 schematically illustrates a beam path of a further embodiment ofa microscopy system 401. The microscopy system comprises an objectivelens 403 having plural lenses 405 and 406. Lenses 405 and 406 arecovered with antireflective coatings such that reflections of visiblelight at the surfaces of the lenses are reduced. The antireflectivecoating may be designed such that also reflections of infrared light andnear infrared light at the lens surfaces are reduced.

Objective lens 403 receives a divergent beam 409 emanating from anobject plane 411 of the objective lens 403. The diverging beam 409 istransformed by the objective lens to provide a substantially parallelbeam downstream of the objective lens. Downstream of the objective lens403 and above the objective lens in the representation of FIG. 4 thereare provided two zoom systems 413 and 414 which are schematicallyindicated in FIG. 4. Each zoom system 413, 414 uses a partial beam 415,416, respectively and supplies the same to oculars 417, 418,respectively of the microscopy system. A user may perceive a magnifiedsharp image of object plane 411 by looking into the oculars 417 and 418with his right and left eyes, respectively. Visible light is used forgenerating these images of the object plane 411. For this purpose, theobject plane 411 is illuminated with visible light supplied by anillumination system 421 comprising a xenon lamp 423 and beam shapinglenses 425 and 426.

The microscopy system 401 further comprises a camera 433 for detecting asubstantially sharp image of the object plane with visible light. Thecamera 433 comprises a CCD camera chip 435 having a light sensitivesubstrate positioned in an image plane 437. A beam splitter 429 isprovided in the partial beam 416 for branching off a beam 431 therefromand for supplying beam 431 to a camera adapter optics 439 supplying thebeam 431 to the camera such that the substantially sharp image of theobject plane 411 is generated at the image plane 437. The imagesdetected by camera 433 may be used for documentation or they may bedisplayed by an display apparatus for displaying the image of the objectplane 411 for users who may not directly use the oculars 417, 418. Theimages of camera 433 may be in particular supplied to a head mounteddisplay of a user.

The microscopy system 401 comprises a camera 441 for detecting images ofthe object plane with infrared light. Camera 441 comprises a CCD camerachip 443 having a light sensitive substrate positioned in an image plane445. A camera adapter optics 447 is provided for supplying a beam 451branched off from the partial beam 451 by a beam splitter 449 to the CCDcamera chip 443. The camera adapter optics 447 is configured such that asubstantially sharp image of the object plane 411 is generated in imageplane 445 with infrared light. Thus, the cameras 433 and 441 differ fromeach other in that camera 433 generates a substantially sharp image ofthe object plane 411 with visible light, and camera 441 generates asubstantially sharp image of the object plane with infrared light.According to one conventional definition the infrared light may comprisewavelengths in a range of 820 nm to 870 nm.

A filter 453 is disposed in beam 451 in front of camera 441. Filter 453is adapted to the fluorescent substance which is used in theapplication. In the present example the filter 453 is adapted to thefluorescence of indocyanine green such that it transmits substantiallyonly light of a wavelength range between 820 nm and 870 nm. Thefluorescent wavelengths of indocyanine green are within this wavelengthrange.

According, to an alternative embodiment the beam splitter 449 may becovered with a suitable coating such that the beam splitter 449 deflectsonly infrared light.

Images detected by camera 441 are supplied to a controller or computer455.

In an application according to one embodiment a tissue to be inspected,such as a human liver is positioned in the object plane 411. Bloodvessels extending through the tissue are substantially not visible ifthe tissue is observed by just using the visible light images providedby oculars 17 and 18. It is not easy to discriminate between bloodvessel and surrounding lever tissue from such images. After anintravenous injection of ICG the fluorescent substance will accumulatein the vessels at a higher concentration than in surrounding tissue. Animage of the tissue using light in the wavelength range of 820 nm to 870nm will show higher intensities at locations corresponding tofluorescent vessels as compared to surrounding tissue.

An example of an image detected by camera 441 and supplied to controller455 is schematically illustrated in FIG. 5 a. A major portion 57 of animaging field 58 shows a very low intensity. A portion 59 shows aslightly higher intensity, and two portions 61, 62 show even higherintensities. Within portion 61 there is located a portion 63 showing aneven higher intensity of infrared radiation. It is assumed that theportions 62 and 63 are associated with blood vessels, whereas theportion 57 is associated with surrounding tissue. It is further assumedthat the portion 59 is associated with surrounding tissue in which somelow concentration of fluorescent substance has accumulated.

The microscopy system 1 further comprises a display system 465comprising an LCD chip 469 positioned in a plane 467. An image displayedwith LCD chip 469 is superimposed with partial beam 415 by a projectionoptics 471 and a beam splitter 473. When looking into the ocular 417 theuser may perceive a superposition of the visible light image of theobject plane and an image representation generated by display 469. Thecontroller 455 may supply an image to display 469 as it is schematicallyillustrated in FIG. 5 a. The image displayed and perceived the user withvisible light of e.g. blue color. Thus, the user is provided with avisible representation of the infrared image in a superposition with thevisible light image. The user may then recognize blood vesselspositioned within the object field of the microscopy system 1.

However, the superposition of the visible light image with the imageaccording to FIG. 5 a would result in a reduction of the informationwhich may be gained from the visible light image within portions 61 and62 since these portions are indicated in blue color. To improve thissituation the controller 455 performs an analysis of the images receivedfrom camera 441. The controller determines those coherent portions ofthe image showing intensities above a predetermined threshold. Using asuitable predetermined threshold a discrimination may be made betweenblood vessels and surrounding tissue. In the example shown in FIG. 5 athe threshold will be adjusted such that the intensity in the portion 59is below the threshold, and such that the intensities within portions 62and 63 are above the threshold.

After identifying the coherent portions exceeding the threshold thecontroller 455 will determine peripheral lines surrounding the coherentregions. Such peripheral lines are associated with a boundary betweenthe coherent portions and the surrounding portions of the image. Thecontroller 455 supplies data representing the peripheral lines to thedisplay 465. The display generates an image of the peripheral lines, andsuch image is superimposed with the visible light image as schematicallyillustrated in FIG. 5 b. In the image, only the peripheral lines 75 ofportions 61 and 62 are shown in blue color. Thus, the user is providedwith the information relating to the blood vessels which are located inan interior of the peripheral lines 75, and the user may still perceivethe visible light image of the blood vessels as usual, and he mayperform a surgical treatment of these blood vessels while observing thevisible light image thereof.

A filter 477 is disposed in a beam path of illumination system 421.Filter 477 is substantially not transmissive for wavelengths of thefluorescent emission of the fluorescent substance. The object will notbe illuminated with fluorescent light such that the fluorescence of thesubstance is visible in the images detected by camera 441 with arelatively high contrast and low background.

Additionally, a filter chopper 479 is disposed in the beam path of theillumination system 421. The filter chopper 479 is rotatably driven by amotor 481 which is controlled by controller 455. The filter choppercomprises plural sectors which are subsequently transmissive andnon-transmissive for light at wavelengths in a range between 750 nm and820 nm. All sectors of the filter chopper 479 are substantiallytransmissive for visible light. The excitation of the fluorescentsubstrate is modulated by rotating the filter chopper 479. Theintensities of the fluorescent images detected by camera 441 aremodulated in time, accordingly, and the controller 455 may analyze thetime dependency of the fluorescent image by a method such as a lock-inmethod for further reducing noise and background in the fluorescentimage.

An alternative embodiment of the illumination system illustrated aboveis indicated by dashed lines in FIG. 4. The alternative illuminationsystem 490 comprises a light source 491 separate from light source 423.Light source 491 is provided for illuminating the object with visiblelight, whereas light source 423 is only provided for generating theexcitation light of the fluorescent substance. Thus, the illuminationwith visible light is independent from the illumination with excitationlight, and a rotation of the chopper wheel 479 may not modulate theillumination with visible light which modulation might disturb the userin observing the visible light image of the object. According to afurther embodiment the light source 423 is a laser light source which israpidly switched on and off by the controller 455 for modulating theexcitation light. The light modulating chopper may be omitted in suchembodiment.

The microscopy system further comprises an optical coherence tomography(OCT) apparatus 200 emitting an analyzing light beam 205 and directingthe analyzing light beam 205 onto a beam scanner 260. Beam scanner 260comprises a mirror for directing the analyzing light beam onto theobject plane 411 and to focus the analyzing light beam 205 onto theobject plane. The beam scanner 260 is controlled by controller 455 forselecting the locations at which the analyzing light beam 205 isdirected onto the object plane and to change those locations. The OCTapparatus 200 detects depth profile data of the object at the selectedlocation and transmits the depth profile data to controller 455. OCTapparatuses are well-known from the art. Examples are given in U.S. Pat.No. 5,493,109 and U.S. Pat. No. 5,795,295, the full disclosure of whichis incorporated herein by reference.

A function of the OCT apparatus 200 is shortly illustrated withreference to FIG. 6 below. The apparatus 200 comprises a white lightsource 220 emitting radiation coupled into an optical fiber 230. A beamcoupler 240 is provided for coupling the radiation into two opticalfibers 250 and 270. One partial beam of fiber 270 is directed onto areference mirror 290 through a lens 280. The partial beam of fiber 250is collimated through a lens 251 as the analyzing light beam 205 anddirected to the beam scanner 260. The beam scanner 260 directs theanalyzing light beam 205 onto the object 255 to be inspected. Radiationof the analyzing light beam 205 received back from the object issupplied by beam scanner 260 in a reverse direction back to the OCTapparatus 250 and coupled into fiber 250. The radiation reflected backfrom mirror 290 is again coupled into fiber 270. The beam coupler 240superimposes the radiation received from the object through fiber 250and the radiation reflected back from mirror 290 through fiber 270 andcouples the superimposed radiations into fiber 265. Fiber 265 suppliesthe superimposed radiation to a photodetector 275. An output of thephotodetector is demodulated by a demodulator 285 and transformed tocomputer readable data by an analog-digital-converter 295 and suppliedto the controller 455.

The detector 275 receiving the radiation from the object 255 and themirror 290 detects a signal which is increased by interference ifoptical wavelengths of the two partial beams between the beam splitter240 and their superposition at the beam splitter 240 are equal within acoherence length of the light source. For achieving such equal opticalbeam paths, the reference mirror 290 is displaceable in a directionindicated with arrow 291 in FIG. 6. By displacing the mirror 290 andrecording the corresponding intensities detected by detector 275 it ispossible to detect a depth profile of the object 255 at that location atwhich the analyzing light beam 205 is directed onto the object 255.Obtaining such depth profile is time consuming since the mirror 290 hasto be mechanically displaced.

The controller 455 controls the beam scanner 260 to direct the analyzinglight beam 205 to those locations on the object at which depths profilesshould be recorded. The controller 455 limits the recording of depthsprofiles to only those portions or analyzing regions which have beenpreviously determined by controller 455 from the fluorescent light imagewhich are indicated by reference numeral 62 and 63 in FIG. 5 b.

The controller 455 controls the beam scanner 260 such that depthprofiles are recorded at a plurality of locations positioned on straightlines 213 within portions 62, 63, wherein the straight lines 213 arevertically arranged in the visible field 58 and disposed at apredetermined distance from each other. The depth profiles recordedalong lines 211 are displayed on a display 207 of the microscopy system401. A keyboard 209 or other input means, such as a mouse, may be usedfor selecting the configuration of the straight lines 213 within thevisible field 58, such as an orientation thereof and distance from eachother. Further, one of portions 62, 63 may be selected such that depthsprofiles for the selected portion are not shown on display 207.

It is also possible to display some selected of the depths profiles byimaging system 65 and to superimpose such representations with thevisible light image perceived through the ocular such that the user mayanalyze the depth profile while observing the image of the objectthrough the ocular.

In the above illustrated embodiments indocyanine green is used as thefluorescent substance. However, other fluorescent substances may be alsoused. In particular, an auto-fluorescence of substances of the humanbody may be observed. As an alternative or in addition to the analysisof intensities of fluorescent images, also fluorescent half times may beanalyzed for discriminating fluorescent regions from each other withinthe object field.

According to further embodiments the image generated by displayapparatus 65 is coupled into partial beam 416 rather than partial beam415. Alternatively corresponding representations may be coupled intoboth partial beams.

In the preceding embodiments the peripheral lines 75 are represented asfull lines. According to further embodiments, the peripheral lines maybe represented as broken lines, dotted lines, dot-dashed lines or othertypes of lines, or the interior of the coherent portions may berepresented as a shaded or hatched region of the image.

Therefore, while the present invention has been shown and describedherein in what is believed to be the most practical and preferredembodiments, it is recognized that departures can be made therefromwithin the scope of the invention, which is therefore not be limited tothe details disclosed herein but is to be accorded the full scope of theclaims so as to embrace any and all equivalent methods and apparatus.

1. A microscopy system for visualizing a fluorescence of a fluorescentsubstance in an object to be inspected, comprising: a microscopy opticshaving a beam path for optically imaging an object region onto a lightdetecting component of a first camera for generating first image datarepresenting images of the object region with light includingwavelengths of a wavelength range comprising a fluorescent emissionwavelength of the fluorescent substance, and a memory for storing a setof first image data generated by the first camera during a timeduration, the first set of image data representing a first time seriesof images of the object region for the time duration; a display system;and a controller for accessing the stored set of first image data fromthe memory, the controller selecting a subset of image data from thefirst set of image data, the subset representing a second time series ofimages of the object region selected from the first time series ofimages of the object region, the second time series of images of theobject region being fewer in number than the first time series of imagesof the object region, the controller supplying plural secondrepresentations generated from the second time series of images of thestored set of first image data to the display system so as to cause thedisplay system to sequentially display the plural second representationsgenerated from the second time series of images of the stored set offirst image data.
 2. The microscopy system according to claim 1, whereinthe beam path comprises at least one light detecting component of asecond camera for generating second image data representing images ofthe object region with visible light, and wherein the display system isfurther configured to display a representation of the second image data.3. The microscopy system of claim 1, wherein the fluorescent substancecomprises indocyanine green.
 4. The microscopy system of claim 1,wherein the controller is configured to cause the display system torepeatedly display the plural second representations generated from thesubset of image data in sequence.
 5. The microscopy system of claim 2,comprising a filter disposed in the first beam path upstream of thefirst camera, wherein the filter is substantially non-transmissive atwavelengths below 810 nm and is substantially transmissive atwavelengths above 810 nm.
 6. The microscopy system of claim 1, whereinthe controller is configured to select the subset of image data of theset of first image data from the set of first image data based onintensities of the images represented by the first image data.
 7. Themicroscopy system of claim 1, wherein the controller is configured forselecting the subset of image data from the set of first image databased on differences between intensities of the images represented bythe first image data.
 8. A microscopy method of visualizing afluorescence of an object to be inspected, the method comprising:detecting a first time series of plural fluorescent light images of theobject during a time period; storing image data corresponding to thefirst time series of plural fluorescent light images in a memory;accessing the stored image data corresponding to the first time seriesof plural fluorescent light images from the memory; selecting a subsetof image data from the stored image data, the subset of image datarepresenting a second time series of plural fluorescent light images ofthe object selected from the first time series of plural fluorescentlight images of the object; the second time series of plural fluorescentlight images of the object being fewer in number than the first timeseries of plural fluorescent light images of the object; and displayingthe second time series of plural fluorescent light images of the objectgenerated from the accessed image data after the time period has lapsed.9. The method of claim 8, wherein the displaying the second time seriesof plural fluorescent light images of the object comprises repeatedlydisplaying the second time series of plural fluorescent light images.10. The method of claim 8, comprising repeatedly displaying in sequencethe series of plural fluorescent light images generated from the subsetof image data accessed from the memory.
 11. A method of treating ananeurysm of a patient, the method comprising: clipping an aneurysm sacof the aneurysm using a clip; injecting indocyanine green into thepatient; detecting a first time series of plural fluorescence images ofat least one artery adjacent to the clipped aneurysm; storing image datacorresponding to the first time series of plural fluorescent lightimages in a memory; accessing the stored image data corresponding to thefirst time series of plural fluorescent light images from the memory;selecting a subset of image data from the stored image data, the subsetof image data representing a second time series of plural fluorescentlight images of the object selected from the first time series of pluralfluorescent light images of the object; the second time series of pluralfluorescent light images of the object being fewer in number than thefirst time series of plural fluorescent light images of the object;displaying the second time series of plural fluorescent light imagesgenerated from the accessed image data such that the second time seriesof plural fluorescent light images is visible for the user; assessingvascular blood flow of the at least one artery based on the second timeseries of plural fluorescent light images; assessing whether theindocyanine green accumulates in the aneurysm sac based on the secondtime series of plural fluorescent light images; and assessing a completeblocking of the aneurysm sac with the clip if the indocyanine green doesnot accumulate in the aneurysm sac.
 12. The method of claim 11,comprising repeatedly displaying in sequence the series of pluralfluorescent light images generated from the subset of image dataaccessed from the memory.